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Abstract —Singapore, an urbanized and populated country 
with high penetration of smartphones, provides an excellent 
base for citizen-centric participatory sensing applications. Mobile 
participatory sensing applications offer an efficient means of 
directing feedback to government agencies for timely identifying 
and solving problems of citizens’ concern. While real deployments 
of such applications are on an uprising trend in Singapore, there 
is no concerted effort that studies the user experience of these 
applications. To fill this gap, we conduct a market study by 
analyzing the user reviews on the Google Play and Apple App 
Store for six major mobile crowdsourcing applications created by 
Singapore government agencies. This study was carried out for a 
period of 4 months during which we collected and analyzed 592 
customer reviews. This was also supplemented by our personal 
use of the applications during the same period. 

This paper presents the methodology and findings of this study, 
as well as our recommendations of what improvements that 
these applications could incorporate. We classify user reviews 
into 8 major concerns, and recommend 9 features to enhance 
the applications’ utility. The recommendations are presented in 
terms of user interface, incentive, and publicity. 

Index Terms —Mobile crowdsourcing, user experience, incen¬ 
tive, trust. 


1. Introduction 

Participatory sensing is a process that tasks mobile de¬ 
vices such as cellular phones to form interactive participatory 
sensor networks that enable public and professional users 
to gather, analyze and share local knowledge Q. There 
are several different usage models of participatory sensing, 
ranging from public contribution, in which individuals collect 
data in response to inquiries defined by a central entity or 
other individuals, to personal use and refiection, in which 
individuals log information about themselves Q. This paper 
focuses on using participatory sensing as a civic tool to gather 
on-the-ground, citizen-initiated feedback that can be shared 
with relevant authorities as well as the rest of the public for 
efficient resolution of issues. 

The high population density and mobile device penetration 
in Singapore presents great potential for participatory sens¬ 
ing to provision pervasive coverage without incurring cost 
of deploying instrumented sensors or specialized workforce. 
Therefore, it can possibly offer a cost-efficient and large-scale 
means of data collection and analysis across various industry 
sectors. 

However, this potential has not been effectively exploited 
so far, according to a recent survey on real deployments of 


participatory sensing Q. In the context of Singapore, while 
there are individual crowdsourcing applications designed for 
government agencies, the concept of participatory sensing 
has not gained much of the public’s attention due to the 
fact that these applications pertain to specific, isolated issues 
such as monitoring litter, traffic or unsafe work practices. 
Convergence of these applications into a unified platform that 
accepts generic issues yet can categorize and direct the issues 
to the relevant government agencies, therefore, would foster a 
larger user base which is critical to the success of participatory 
sensing. 

While much effort has been invested in developing par¬ 
ticipatory sensing applications, there is no concerted attempt 
to investigate user experience and lessons learnt from these 
applications. In this paper, we present a market research study 
we have conducted on extensive user reviews of a wide range 
of existing applications. We have also used all the applications 
in person for an extended period of time and incorporated 
our own experience into this study. Eurthermore, based on the 
analysis of our findings, we recommend features and schemes 
that should enhance the application’s utility and appeal to 
users. 

We also refer to some prior work conducted in various 
contexts as a guideline in planning and organizing our study 
as well as analyzing the market data. Eor example, 0 derives 
some qualitative aspects of introducing mobile reporting, as 
shown in Eig. 0 

The rest of the paper is organized as follows. We describe 
our methodology in Section and summarize our findings in 
Section III We then present our recommendations in Section 
|IV[ Einally, we conclude with possible directions for future 
research in Section m 


II. Methodology 

This market research was conducted in three stages. The first 
stage involves background study and identifying prominent 
existing crowdsourcing applications to analyze. The second 
stage comprises collecting and analyzing user reviews of these 
applications at an extensive scale, as well as our personal 
usage of the applications. In the last stage, the user review data 
was categorized in a way that facilitates identifying larger and 
more general groups of concerns that consumers may have, by 
which we zoom out from micro to macro scope and provide 
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User aspects 

Process and 
organization aspects 

Privacy and 
legal aspects 

Technical aspects 

• Easy interaction 

• Issue tracking 

• Improved 
communication 

• Exceeded 
expectations 

• Increased 
satisfaction 

• Increased report 
quality 

• Reduced throughput 
times 

• Accident prevention 

• Analytics 

• Routing across 
municipalities 

• Needed citizen data 

• Defined terms of use 

• Deletion after closing 

• Transparency issue 

• Photo third party rights 

• Liability in imminent 
danger 

• Message filtering 

• Broadband coverage 

• Less media disruptions 

• Integration with ticketing 
system 


Fig. 1. Qualitative aspects of introducing mobile reporting. 0 


recommendations that can be incorporated into both current 
and potentially future applications for various enhancements. 

A. Stage 1: Background Research and Identification of Rele¬ 
vant Applications 

To ensure relevance to the development of participatory 
sensing applications, we had carefully selected a pool of 
mobile applications for our consumer review analysis. The 
basic criteria include (1) the applications were created by 
a Singaporean government agency rather than by private 
institutions or individual developers, because we theme around 
sustainable city development of Singapore and we also require 
the quality of the applications to be properly regulated, (2) 
the applications need to involve some form of crowdsourcing 
as a major element, and (3) the user base should be sizable 
and there are a considerable volume of customer reviews 
available, since these indicate the popularity of an application. 
Eventually, we were able to include in out study not only the 
most favored applications in terms of user reviews but also the 
least liked applications. This allows us to understand both the 
desirable features of consumers as well as their turnoffs. On 
our final list are six crowdsourcing applications: MyEnv 
CleanLah Q, and WeatherLah 0 by the National Environ¬ 
mental Agency (NEA), Snap@MOM by the Ministry of 
Manpower (MOM), My Transport by the Land Transport 

Authority (ETA), and Police@SG | [10| by the Singapore Police 
Office. 

B. Stage 2: Verification, Analysis and Categorization of Cus¬ 
tomer Reviews 

The purpose of this stage was to gain insights into the 
public opinion on the pros and cons of current crowdsourcing 
applications. Besides analyzing existing user reviews, it is also 
a key to gain first-hand experience. Therefore, we downloaded 
them into our own phones, Samsung Galaxy Grand, Motorola 
MB525, and iPhone 4, which run the two most popular 
mobile OS (Android and iOS), and used them in person for 4 
consecutive months. 

We created a database for each of the participatory sensing 
applications under investigation. Then, we read and verified 
all the user comments on the Playstore and Appstore for 
these applications, and populated the database by importing 
the comments into the database separately as per whether they 
are positive or negative comments. 


To ensure the relevancy and accuracy of user comments, 
we also verified them carefully based on the latest updated 
version of the pertaining application. Eor example, if a user 
commented that there was no way to turn off notifications but 
there actually was, this comment will instead be categorized 
as confusing interface design because the user did not manage 
to locate the notifications toggle button. If a comment is based 
on an early version of the application that does not have 
the notification toggle button, while the latest version does 
have, the comment will be deemed as invalid or outdated 
and not imported into the database. The above verification 
process also took into account the different contexts of the 
comments in a case-by-case manner. The comments were 
finally categorized into broader groups of user concerns for 
each particular application, with a number indicating the size 
of each group. 

In total, we have collected and analyzed 592 user comments 
on Google Play and Apple App Store over a period of four 
months (August-December 2013). Due to the casualness of 
users writing reviews, we filtered out many uninformative 
or invalid comments, and eventually selected 281 useful re¬ 
views to compile our findings]^ Eurthermore, we have also 
augmented this user review database with our own personal 
use of the applications on both Android and iOS devices over 
the same period of four months. 

C. Stage 3: Cross-Application Generalization 

In this stage, we generalize the above application-specific 
categories to wider user concerns across all these applications. 
The purpose was to create a guideline for crowdsourcing 
application designers and developers to refer to, particularly 
for better user experience and a broader outreach. Ultimately, 
we provide an extensive list of recommendations that are appli¬ 
cable to a wide range of crowdsourcing applications, covering 
several important aspects in system design, implementation, 
and deployment. 

III. Eindings 

This section summarizes our findings. We categorize the 
281 user reviews into 8 major concerned areas as well as a 
miscellaneous category, as shown in Table. 

Tn some not-so-common cases, a user commented on different aspects of 
an application, in which case each aspect was counted as a separate comment 
and categorized accordingly. 
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TABLE I 

Summary of User Reviews by Category 


Category 

No. of Vi 

Positive 

ier Reviews 

Negative 

% of Total 

Accuracy and Usefulness of Information 

32 

52 

29.9 

User Interface Design 

20 

48 

24.2 

Compatibility Issues, Responsiveness and Robustness 

3 

56 

21.0 

Quality of Alerts, Notifications and Widgets 

4 

14 

6.41 

Governments Response and Eurther action 

4 

10 

4.98 

Purpose of Application 

1 

13 

4.98 

Quality of Application Updates 

8 

0 

2.85 

Network Connection Issues 

0 

5 

1.78 

Miscellaneous 

2 

9 

3.91 

Total 

74 

207 

100 


A. Accuracy and Usefulness of Information 

This is an often mentioned user concern found in 29.9% 
of all the user reviews. A lack of reliable and accurate 
information is a substantial and non-trivial issue that faces 
crowdsourcing applications. There needs to be a good mecha¬ 
nism that can determine the validity and trustworthiness GD 
of crowdsourced information and sieve out fake and malicious 
information. There is also a need for sufficient deterrents to 
preemptively prevent users from posting false data. Besides, 
the usefulness of information often differs from individual to 
individual; a mechanism that allows a user to personalize the 
application such that he only see information useful to himself 
would therefore be a great advantage. 

B. User Interface Design 

User interface design is the second largest category due 
to its permeating presence throughout every application. This 
was corroborated by our findings, where 24.2% of reviews 
pertained to user interface design. There are three main sub¬ 
categories (see Table. 0 that users are specifically concerned 
about: ease of use, aesthetic appeal, and personalization. 


Aesthetic Appeal. While solid functionality is definitely 
a necessity, aesthetic appeal draws the user to the 
application—about one third of user comments about UI 
concern with aesthetic appeal, highlighting its importance. 
Users prefer applications that use bold and vibrant colors 
rather than plain ones; yet these applications should not 
be cluttered either. A positive example in this respect is 
Snap@MOM (see Fig.|^. In addition, the UI design also 
needs to take into account the tastes of the target user 
group of the application. MyEnv is such an example 
where Android users were dissatisfied with an interface 
ported from iOS while they preferred a Holo-style design. 
Personalization: Applications should be customizable to 
users’ unique preferences—while there are a great number 
of applications which allow users to select parameters for 
the notifications and alerts they receive, the main interface, 
however, is sometimes rather rigid in that users are not able 
to personalize it according to their preferences. Incorpo¬ 
rating this feature would make the application simpler to 
use. MyEnv represents one of the applications that allows 


users to select the main UI they want to use; see Eig. 2c 


TABLE II 

User Reviews about User Intereace Design 


Aspects of UI 

Ease of Use 

Aesthetic Appeal 

Personalization 

No. of Reviews 

32 

33 
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Ease of Use: It is important to users that “the client 
application of a mobile reporting service should possess 
high usability and enable easy interaction to outmatch tra¬ 
ditional channels of communication” 0. It is often helpful 
for applications to provide users with enough options to 
cater for different individuals’ various needs; however, at 
times, a user has to work through a mesh of different 
options to locate what he really needs. As a result, words 
such as “confusing” and “disorganized” appear frequently 
in the reviews of certain applications. Reviews also suggest 
that users tend to favor macro-scale pictorial representation 
that readily gives an intuitive big picture as well as 
supplements detailed specific information; Fig. 1^ gives 
such an example, where WeatherLah’s map presentation 
gained compliments from many user reviews. 


C. Compatibility Issues, Responsiveness and Robustness 

From our own experience on the Motorola MB525 An¬ 
droid phone which is a relatively old model, many of the 
applications, particularly MyEnv and WeatherLah, often ran 
very slowly or even suddenly crash. These compatibility, 
responsiveness HD, and robustness issue of such applications 
was concerned by 5% of user reviews and we deem it should 
not be taken lightly and is worth designers and developers’ 
effort to improve. 

D. Quality of Alerts, Notifications and Widgets 

While alerts, notifications and widgets enable a user to 
receive important information on the go, 6.41% of all user 
reviews and 6.76% of all negative reviews mentioned that 
many notifications are not sufficiently relevant to their needs 
and some are way too frequent and annoying. Therefore, 
in order to accommodate different user needs, notifications 
should be highly customizable to cater to users’ priorities 
and convenience. Some users also encountered difficulty in 
navigating through a complex interface to toggle notifications. 
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Nowcast Crowdcast Drains 



Forecast Crowdcast Map Alerts Twitter 


(a) WeatherLah’s map presentation allows users 
to view weather throughout the whole island of 
Singapore and has been complimented by many 
user reviews. 



(b) Snap®MOM has vibrantly colored illustra¬ 
tions in a simple user interface. 





Selection of Favourite Screen 

Main Menu 


Air Quality 

© 

Weather 

0 

Water Level Sensors 

0 

Dengue 

© 

Hawkers 

© 

Fvpnt.q 

JBl 


(c) MyEnv has the option to allow users to see 
different screens upon opening the application, 
according to their preference. 


Fig. 2. Representative user interfaces. 


hence hinting toward the need for an intuitive user interface 
design. 

E. Governments response and further action 

The applications CleanLah and Snap®MOM, which func¬ 
tion by relying on participants feedback, received a large 
percentage (35.7% and 69.2% for these two applications, 
respectively) of user comments that mentioned government’s 
response as a critical factor. These reviews typically concerned 
with the speed and effectiveness of response as well as 
whether users are informed of the follow-up to their feedback. 
Applications thus need to be able to channel information in 
both directions, from users to government as well as from 
government to users. Information updates about the resolution 
of issues, as well as statistics that illustrate adequate responses, 
would therefore be desirable. 

F. Other User Concerns 

Besides the above 5 user concerns which constitute 86.5% 
of user reviews, the following 3 categories add up to only 
9.6% of user comments. However, we deem them to be worth 
considering due to their relevancy to user experience and based 
on our own personal usage as well. 

• Purpose of Application: Some users were unclear of the 
purpose of certain applications (MyEnv, MyTransport, 
Police @SG). This is likely because they were bombarded 
with a lot of information not well organized but cluttered 
in the application. 


• Quality of Application Updates: User reviews indicated 
that the application should not stagnate but improve over 
time by releasing timely updates. On the other hand, the 
updates should not compromise the smooth functioning of 
the application and hence should be released only when 
changes are nontrivial. 

• Network Connection Issues: Some users experienced prob¬ 
lems of Internet connections when accessing or using the 
applications, especially when they are traveling in vehicles. 
This suggests that developers may need to consider other 
communication options such as SMS or MMS. 

IV. Recommendations 

In this section, we provide relevant recommendations which 
would improve participatory sensing applications over the 
issues expressed in the user reviews. When preparing these rec¬ 
ommendations, we took into account public user reviews, our 
personal usage of the applications, and the qualitative analysis 
of participatory sensing from Fig. These recommendations 
are classified into 3 broad categories, namely user interface 
design, incentive, and publicity. 

A. User Interface 

Existing UI can be improved in terms of clarity, accuracy, 
and accessibility. 

1) Dual-Interface System 

Participatory sensing applications that source for user 
feedback rarely allow for two-way communication in 
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that government agencies are not able to communicate 
with users, which is however important because involving 
government agencies to communicate to users within the 
application itself would facilitate quick dissemination of 
information as well as attracting a larger user base and 
user confidence in the application. A system that allows 
government agency to log in (probably with a separate 
interface, such as Snap@MOM illustrated by Fig.|^ would 
make this viable. This would also bolster follow-up actions 
such as government responses, and allows government 
agencies to play a role in, e.g., severity rating. 
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Fig. 3. Snap®MOM allows users of different roles to log in with different 
permissions. 

2) Quality and Validation 

As the quality of contribution (H) is critical to partici¬ 
patory sensing, validation becomes an important way to 
prevent users from being misled by false or inaccurate 
reports from other users. A possible solution is to design 
truth-finding or information-validation algorithms using 
machine learning techniques. Another way is to allow users 
to rate each other’s reports in terms of quality or validity. 

3) Alternative Communication Channels 

Most participatory sensing applications require an Inter¬ 
net connection to send user feedback to the application 
database. However, a stable Internet connection is not 
always available when users are on the go. Therefore, 
having an option in the application to send the information 
through messaging via telco networks would be a solution 
worth consideration. To this end, a special telco number 
needs to be registered with a telco. 

4) Widgets 

This recommendation specifically pertains to Android 
phones. Widgets allow for user-specific information to be 


easily accessed and quickly seen by the user on the home 
screen of his phone. Users can prioritize on which type 
of information to be displayed on the widget and specify 
what frequency the information should be updated. Several 
applications have an option to activate widgets, but lack 
the flexibility of personalizing them to users’ own needs 
and interests, e.g., by relating to a certain residential area 
or a certain type of issues. 

5) Severity Rating 

The owner of a post would be allowed to assign a severity 
value so as to indicate the importance or urgency of the 
pertaining problem. On the other hand, as it only refiects 
the subjective view of the owner, the application may also 
allow other users to add their ratings in order to calculate 
an average which bears more objectiveness. In addition, the 
government agency may also be allowed to rate the issue 
for greater conformity and less subjectivity. An aggregation 
algorithm could be designed to take into account different 
weights of different users and government agencies. 

6) Pictorial Representation 

A macro-scale pictorial representation enhances clarity by 
providing users who do not want to read text-based details 
with a quick and easy way to get an overall sense, such 
as all problem occurrences in the region together with 
their nature and severity. Fig. [^illustrates this idea. Also, 
problems may be color coded (e.g. according to severity 
rating) or with different icons (e.g., garbage representing a 
cleanliness problem while a vehicle representing a traffic 
problem), and users may be able to set a filter to see 
problems of his interest only. 



Fig. 4. Pictorial representation as in a map. 


B. Incentive 

Incentives are key to make participatory sensing applications 
successful because these applications completely rely on the 
public users’ participation in order to collect data or feedback. 

7) Incentive mechanisms 

Incentive mechanisms are an effective and theoretically 
proven tool to motivate users to make contributions in 
participatory sensing. There is a rich literature dwelling on 
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this; for example 0 proposed an all-pay auction based 
incentive mechanism and could be referred to. 

8) User ranking 

Employing a weekly/monthly/yearly leaderboard system 
would provide recognition to committed users and serve as 
motivation to continually posting. Each post should earn 
points depending on its spatiotemporal property, impor¬ 
tance, urgency, etc. The leaderboard thus may comprise 
those who have earned the most number of points within 
a certain period of time. In a sense, it also adds some gam- 
ification element and fun competition to the application to 
appeal to users. 

C. Publicity 

Publicity can be enhanced by capitalizing prominent exist¬ 
ing platforms. 

9) Social Media 

The advent of Eacebook has made waves around the 
world with 1.2 billion users in just 8 -f years. This predis¬ 
posed popularity of social media over the world populace 
presents social media as an effective platform to boost 
a larger user base for crowdsourcing. Using social me¬ 
dia like Eacebook and Twitter as a publicity mechanism 
would provide users of participatory sensing applications 
an option to share their posts on particular issues on other 
popular channels. Eurthermore, enabling the penetration of 
information in this way also helps gain user support from 
a larger community. 

10) Accompanied Website 

A website specifically designed for a mobile application 
can serve to publicize the application as well as incentivize 
users to use the application by providing statistics, figures 
and pictures, without cluttering up the phone interface. It 
allows users to better understand the application’s purpose 
and features. The site also helps involve users to a deeper 
extent, by including a page where users can share feedback 
about the application, and the feedback can be channeled 
to the developers. In fact, it becomes a vehicle for devel¬ 
opers to crowdsource ideas and suggestions from users to 
incorporate into the applications. 

V. Conclusion 

In this paper, we present an extensive market study we 
have conducted on government-centric participatory sensing 
applications in Singapore, with the purpose of understanding 
the desirable features for sustained use of these applications. 
Our study has identified 8 major user concerned areas and 
provided 9 relevant recommendations for both existing and 
future participatory sensing applications. 

Eurther scope for research would include the use of machine 
learning techniques to facilitate analyzing the data sets, which 
was carried out manually in this work. That way, we should 
achieve much greater efficiency and be able to analyze a much 
larger set of customer reviews as well as other data sources. 
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